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System And Method For Providing A Visual Language 
For Non-Reading Sighted Persons 

5 [0001] This Application claims priority from Provisional Patent Applications 
Serial Nos. 60/418,533, filed October 15, 2002. and 60/459.063, filed March 28. 
2003. 

Field Of The Invention 

[0002] The present invention generally relates to communications systems 
10 for non-reading sighted persons and, more particularly, to a system and method 
for providing a visual language for non-reading sighted persons such as the 
cognitively disabled. 



Background Of The Invention 

15 [0003] The Braille format was devised to enable the blind or visually 
impaired to read alphanumeric characters using their sense of touch. Braille is 
presented in "cells" of dots raised above the surface of the reading material in 
unique patterns so that a sight-impaired person may simply brush their fingers 
across the field of dots to feel the pattern of dots and thereby "read" the letters 

20 represented therein. Initially each Braille cell consisted of six dots, arranged in two 

columns of three dots each. Each alphanumeric character is represented by a 

unique set or pattern of six raised dots in the cell. Because of the number of 

combinations possible in a six-dot cell is limited to sixty-three, the cell has been 
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expanded to eight dots, by adding two dots at the bottom, to form two columns of 
four dots. Braille recognition technology is known in the art that optically scans the 
projecting Braille dots, recognizes the pattern, and associates it with a particular 
alphanumeric character, via a computer, so that Braille texts may be read via the 
5 computer's user interface to persons without sight. Braille code on printed matter 
is often referred to as being in "India-ink". U.S. Patents Nos. 5,243,655; 
5,982,911; 5,710,417; 5,169,342, and 6,625,613 disclose various devices and 
methods for recognizing and interpreting Braille text. Braille does not aid the 
sighted person who cannot read alphanumeric text do to illiteracy or a cognitive 
10 disability. 

[0004] Bar code labels are known to comprise a series of parallel dark bars 
of varying widths with intervening light spaces, also of varying widths. The 
information encoded in the bar code is represented by the specific sequence of 
bar and space widths, the precise nature of this representation depending on the 

15 particular bar code symbology used. Methods for reading bar codes may comprise 
generation of an electronic signal wherein a signal voltage alternates between two 
preset voltage levels, one representing a dark bar and the other representing a 
light space. The temporal widths of these alternating pulses of high and low 
voltage levels correspond to the spatial widths of the bars and spaces. It is this 

20 temporal sequence of alternating voltage pulses of varying widths which is 

presented to an electronic decoding apparatus for decoding. Prior art bar code 
scanners are typically operated in one of two modes, fixed or handheld. In the 
fixed mode of operation, objects with bar codes thereon are moved to or past a 
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stationary bar code scanner for scanning. In tfie handlield mode of operation, a 
portable bar code scanner is typically oriented and/or moved to the bar code label 
to be read. Examples of various bar codes and bar code readers are shown in 
U.S. Pat. Nos. 3,918,029; 4.275,381; 4.283 622; 4,323.772; and 4,329.574. Bar 
5 codes are not used in combination with one another to form symbolic 

representations of thought or alphanumeric text, nor can they be used effectively 
to communicate instructions to the sighted person who cannot read alphanumeric 
text do to illiteracy or a cognitive disability. 

[0005] Non-reading sighted persons may be illiterate or be afflicted with a 
10 cognitive disability that prevents or impedes their ability to receive information 
through reading standard alphanumeric text or Braille code. Braille was designed 
and intended to be read by sightless persons, while bar code symbologies were 
designed to be machine generated and machine readable. In the case of bar 
codes, they must be printed with standard alphanumeric text (or Braille code) in 
15 order for the underlying information to be readable to a person. 

[0006] Graphic languages have been used to communicate with language 
deficient persons, and graphic user interfaces have been the standard for user 
interfaces on personal computers and the internet. Braille has not been translated 
to graphics for persons with sight who can not read text, nor have bar code 
20 technologies been used to facilitate communication between humans and 
machine, or to enhance human reading ability. 
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[0007] As a consequence, there has been a long felt need for a graphic 
language combined with machine vision technologies to generate a new visual 
language for non-reading sighted persons such as the cognitively disabled. 



5 

Summarv Of The Invention 

[0008] The present invention provides a method and system for 
communicating to sighted persons who cannot read a standard alphanumeric text, 
the informational content of that text. In its most general aspects, a method is 

10 provided in which an alphanumeric text is converted to a machine-readable set of 
informationally corresponding data, and then scanned or otherwise read so as to 
obtain an informational content data set. The informational content data set is 
then correlated with one or more hieroglyphs that are representative of the actions, 
physical objects and the relationship between the actions and the physical objects 

15 presented by the infomriationally con-esponding data set. The one or more 
hieroglyphs are then syntactically arranged to pictorially represent the 
informationally corresponding data set for a sighted person thereby to inform them 
of the actions, physical objects and the relationship between those actions and 
physical objects as was originally presented in the standard alphanumeric text. 

20 [0009] The present invention also provides a system for converting 

alphanumeric text into a machine readable Braille, reading that machine readable 

Braille, and then using a relational database to associate the infomiation content 

of the machine readable Braille with one or more conventionalized pictures or 
Docket No.: H2035-00005 -4- 



symbols (hieroglyphs) that are representative of actions, physical objects and the 
relationship between the actions and the physical objects presented by the 
alphanumeric text so as to syntactically arrange and correlate a series of the 
hieroglyphs to pictorially represent the actions, physical objects and the 
5 relationship between those actions and physical objects as was originally 
presented in the standard alphanumeric text. 

Brief Description Of The Drawings 

[001 0] These and other features and advantages of the present invention 
10 will be more fully disclosed in, or rendered obvious by, the following detailed 

description of the preferred embodiment of the invention, which is to be considered 
together with the accompanying drawings wherein like numbers refer to like parts 
and further wherein: 

[0011] Figs. 1 is a schematic representation of a portion of a system for 
15 converting alphanumeric text into a machine readable Braille in accordance with 
an embodiment of the invention; 

[0012] Fig. 2 is a perspective view of an optical scanner reading machine 
readable Braille in accordance with an embodiment of the invention; 

[0013] Fig. 3 is a schematic representation of one process for using a 
20 relational database to associate the information content of the machine readable 
Braille with one or more conventionalized hieroglyphs; 
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[0014] Fig. 4 is an Illustration of one or more conventionalized hieroglyphs 
that are representative of actions, physical objects and the relationship between 
the actions and the physical objects presented by the alphanumeric text that may 
be syntactically arranged to pictorially represent the actions, physical objects and 
the relationship between those actions and physical objects as was originally 
presented in the standard alphanumeric text; 

[0015] Fig. 5 is a block diagram of a system formed in accordance with the 
present invention for creating an information content data set from an 
alphanumeric text; 

[001 6] Fig. 6 is a perspective view of an encoding means of the system of 
Braille code; 

[0017] Fig. 7 is a representation of one embodiment of a dot-square matrix 
of the type contemplated for use with the present invention; 

[0018] Fig. 8 is flow chart of the encoding process for Braille code; 

[0019] Fig. 9 is a schematic view of a scanner reading a sheet of paper 
containing machine readable Braille; 

[0020] Fig. 10 is a schematic representation of a first portion of a decoding 
and recognition system formed in accordance with the present invention; 

[0021] Fig. 1 1 is a flow chart of the decoding process for Braille code; 
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[0022] Fig, 12 is a schematic diagram of a scanning and decoding system 
forming a hieroglyph for presentation to an end-user in accordance with the 
present invention; 

[0023] Fig. 13 is an illustration of graphic vocabulary for Symbolic reading 
5 system; 

[0024] Fig. 14 continues the illustration of graphic vocabulary for Symbolic 
reading system; 

[0025] Fig. 1 5 is a flow chart of code & symbology interpreting system in 
accordance with one embodiment of the present invention; 

10 [0026] Fig. 16 is a flow chart of one embodiment of a task module for code 
& symbology interpreting system; 

[0027] Fig. 17 is a flow chart of hand held scanner error handling for code & 
symbology interpreting system; 

[0028] Fig. 18 is a flow chart of one embodiment of the display graphics 
15 process for code & symbology interpreting system; 

[0029] Fig. 19 is a flow chart of relational database structures for code & 
symbology interpreting system; and 

[0030] Fig. 20 is a flow chart an embodiment of the Count Processor for the 
for code & symbology interpreting system. 

20 
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Detailed Description Of Preferred Embodiments 

[0031] This description of preferred embodiments is intended to be read in 
connection witli the accompanying drawings, which are to be considered part of 
the entire written description of this invention. The drawing figures are not 
5 necessarily to scale and certain features of the invention may be shown 

exaggerated in scale or in somewhat schematic form in the interest of clarity and 
conciseness. In the claims, means-plus-function clauses are intended to cover the 
structures described, suggested, or rendered obvious by the written description or 
drawings for performing the recited function, including not only structural 
10 equivalents but also equivalent structures. 

[0032] Referring to Figs. 1-7, the present invention provides a 
communication method and system 5 for communicating with sighted persons who 
cannot read standard alphanumeric text 8. In its most general aspects, 
alphanumeric text 8 is converted to a machine-readable set of informationally 

15 corresponding data 10, and then scanned or otherwise read so as to obtain an 
informational content data set 12. Informational content data set 12 is then 
correlated with one or more hieroglyphs 19 that are representative of the actions, 
physical objects and the relationship between the actions and the physical objects 
presented by informationally corresponding data set 12. Hieroglyphs 19 may also 

20 include labels and signs. One or more hieroglyphs 19 are syntactically arranged 
to pictorially represent informationally corresponding data set 12 for a sighted 
person so as to inform them of the actions, physical objects and the relationship 
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between those actions and physical objects as was originally presented in 
standard alphanumeric text 8. 

[0033] A system according to the invention provides for converting 
alphanumeric text 8 into a machine readable Braille (corresponding to 
5 informationally corresponding data 10) reading that machine readable Braille 10, 
and then using a relational database 20 to associate informationally corresponding 
data set 12 with one or more hieroglyphs 19 that are representative of actions, 
physical objects and the relationship between the actions and the physical objects 
presented by the alphanumeric text 8. In one embodiment, the invention 

10 comprises a Braille code symbology 16, code & symbology interpreting software 
13, and a comic & sequential art symbology (hieroglyphs) 19. More particularly. 
Braille code symbology 16 comprises a two dimensional pattern of light and dark 
dots arranged according to conventional Braille code, e.g., a plurality of unique 
patterns of black and white dots that are organized in a six-cell matrix 21 so as to 

15 represent alphanumeric characters, and that are each suitable for recognition as a 
binary character. Conventional Braille code has a different pattern of raised 
bumps for each alphanumeric character. Braille symbology 16 provides a 
machine-readable pattern, i.e., flat, that can be read by, e.g., an optical scanner, 
and thereby deliver all of the information originally presented by the corresponding 

20 alphanumeric text 8. 

[0034] The information encoded in six-cell matrix 21, e.g., a single 
alphanumeric symbol, is represented by the specific pattern of black dots and 
white dots, the precise nature of this representation depending upon standard 
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Braille code symbology. Methods for reading Braille symbology 16 may comprise 
generation of an electronic signal wherein a signal voltage alternates between two 
preset voltage levels, one representing a black dot and the other representing a 
white dot. These altemating pulses of high and low voltage levels correspond to 
5 the presence or absence of black and white dots in a scan line. It Is this sequence 
of alternating voltage pulses which is presented to an electronic decoding 
apparatus for decoding and creation of informationally corresponding data set 12. 
Binary detection by an optical scanner is facilitated by adopting matrices 21 of 
black or white dot-squares that represent one and zero, and are arranged 
10 according to the conventional Braille patterns. A plurality of six-cell matrices 21 
are provided based upon a standard set of ASCII characters, and can be printed 
by conventional Inkjet and laser printers as Braille code symbology 16. 

[0035] Referring to Figs. 1 , 2, and 7, in one embodiment of the present 
invention a Braille code symbology 16 is provided in the form of "India-ink" Braille 

15 code symbology 1 6 which are optically scanable and thereby machine-readable. 
The system often includes data entering means 100, encoding means 102, 
recognition means 108, and decoding means 110. Data entering means 100, e.g., 
a keypad, optical scanner, or magnetic card reader, are arranged in signal 
communication with encoding means 102 so as to create media 106 that display 

20 one or more cell matrices 21 of Braille code symbology 16. Recognition means 
108 scans Braille code symbology 16 and produces one or more output signals 
that replicate the data encoded by encoding means 102. Recognition means 108 
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convert Braille code symboiogy 16, via decoding means 109 into infomnationai 
content data set 12. 



[0036] A processing device 1 12, e.g., a personal computer, PDA, or the lil<e 
computing device of ttie type l<nown in the art, may be used in conjunction with 
5 encoding means 102 to encode alphanumeric text 8 as a plurality of matrices 21 . 
Of course, it will be understood that data to be encoded may be retrieved from 
computer memory files as well as being entered by data entering means 100. A 
display unit 1 1 1 may be employed, such as liquid crystal display or a CRT. In one 
embodiment, processing device 112 generates print signals for controlling a 
10 conventional printer 103 so as to print matrices 21 onto media 106, e.g., a sheet of 
paper. Printer 103 may also print Braille code symboiogy 16 with magnetic-ink so 
that it may be recognized by magnetic-ink recognition sensors forming a portion of 
recognition means 108. 

[0037] Referring to Fig. 7, a typical dot matrix 21 of the type contemplated 

15 for use with the present invention (often referred to as a "codeword" 114) 

comprises two columns of dot-squares 1 18 by three rows of dot-squares 118 

fomriing a single character unit or codeword 1 14. Each individual dot-square 118 

may be identified by a reference character a1 , a2, a3, a4, a5, a6 for ease of 

manipulation as a portion of the data analysis software program. Each character 

20 of alphanumeric text 8 to be read may be represented as a unique combination of 

dot-squares 118, i.e., a unique combination of one to six black dot-squares and 

one to six bordered white dot-squares. A lateral spacing W1 is defined between 

adjacent dot-squares a1 ,a4 a2,a5 and a3,a6 for each codeword 1 14. A 
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longitudinal spacing W2 is defined between a6 and a5, a5 and a4, a3 and a2, and 
a2 and a1 for each codeword 1 14. Braille characters are read from left to right so 
that a string of codewords 1 14 (matrices 21) may be positioned adjacent to one 
another with a start pattern 1 1 5 and a stop pattern 1 1 6 positioned therebetween to 
5 indicate the beginning and end of each string of codewords 1 14. 

[0038] Encoding means 102 processes alphanumeric text 8 by entering it 
into the system using entering means 100 (Fig. 9). A request 130 is sent from a 
Processor Manager 216 (Fig. 15) to get the data to be encoded. The data is 
tested against a specified text image length requirement 132. The factors 

10 determining the image length requirement include, but are not limited to, the dpi 
resolution and capabilities of printing means 103. If the test fails, encode request 
130 is denied and returned to encoding means 100, along with a notice to 
Processor Manager 216. If the test is successful, the encoding processing 
continues by adding two bit Braille start pattern Code word 1 1 5 to encoded scan 

15 data 133 then the remaining number of data character symbols 134 are compared 
with standard Braille character symbols in section 135. The data to be encoded is 
compared in section .135 with standard codeword values. Comparison test 137 
determines whether a valid match is found in section 137. If comparison test 137 
fails, the request is denied and section 143 returns the data to Processor Manager 

20 21 6 as invalid. If comparison test section 1 37 test is successful, the encoding 

process continues to create a codeword 1 14 at section 139. The data is evaluated 
for additional character symbols at section 138. If the evaluation yields no more 
character symbols, a stop pattern codeword 1 16 is inserted at section 142 and 
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encoded data 141 is returned to processor manager 216. If the evaluation yields 
additional character symbols, the process loops at section 134 to continue the 
encoding of the additional character symbols until this loop terminates and returns 
to Processor Manager 216. 

[0039] Referring to Fig. 9, recognition means 108 often comprises a 
handheld scanning unit 145 comprising a scanner with a digital signal processor, 
DSP, and a handheld computer. A display screen 147 may be included, and may 
comprise a liquid crystal display, touch receptive pad, or a scan trigger 146 as an 
input means. Handheld unit 145 has a two dimensional image processing 
capability and is capable of recognizing and decoding Braille code symbology 16. 

[0040] Referring to Fig. 10, an electronic imaging and decoding process 
using a handheld scanner 145 with a digital signal processor (DSP) and a 
handheld computer in accordance with one embodiment of the invention starts 
when an end-user activates a scan trigger 146 to start the electronic imaging 
process. Scan trigger 146 initiates a request to scan processor 156 to launch 
illumination system 157 which illuminates target image 159 (i.e., a sheet of paper 
containing Braille code symbology 16). Light is reflected from image 159 through 
a lens system and focused onto two dimensional imager 162. Two dimensional 
imager 162 sends image data to interface imager 163. The image data is stored in 
a device memory 164. A scan processor 156 retrieves image data from device 
memory 164 and decodes the image data using a scan decode process 165 which 
returns decoded scanner data to handheld CPU 167, where it is graphically and 
syntactically Interpreted for end-users 203 with the code & symbology interpreting 
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software 13. Illumination system 157 may include an LED or Visible-sight laser 
Diode. In other embodiments, two dimensional imager 162 may include a CCD 
imager. Illumination system 157 may also include a flash lamp. 



[0041] Code & symbology interpreting software 13 interprets scanned 
5 Braille code symbology 16 as an informationally corresponding data set 10, and 
converts that data set into syntactically corresponding visual nan^ative comprising 
one or more hieroglyphs 19 that sighted persons who cannot read a standard 
alphanumeric text can interpret. In this way, these persons will be assisted in 
navigating their physical environment or completing needed tasks in a work 
10 environment. More particularly, code & symbology interpreting software 13 cross- 
references informationally corresponding data set 10 with a relational database 
that stores a wide variety of hieroglyphs 19 that may be presented to a sighted 
person who cannot read a standard alphanumeric text as symbolic representations 
of nouns and verbs in graphic form. 

15 [0042] Referring to Fig, 1 1 , the decoding and interpretation process starts 
with the transmission of image data (I.e., infomiationally con-esponding data set 
12) from device memory 164 to scan processor 156 as decoded scanner data 166 
(Fig. 7). The image data is analyzed by testing its length against a specified text 
length requirement 172, e.g. a length equal to the number of characters plus four, 

20 but less than a text limit. The factors determining the image length requirement 

specified include, but are not limited to, pixel resolution and capabilities of 

handheld unit 145. If length test 172 fails, then an invalid code error 180 is 

returned to scan processor 156. If the test succeeds, the image data Is tested at 
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section 1 73 for a start pattern 1 1 5, e.g., the first two bits are checked. If the test 
fails, a reverse order process 177 starts. The image data is once again tested for 
start pattern 115 in section 181. If the test at section 181 fails, an error invalid 
start pattern 1 86 is returned to scan processor 1 56. If the test succeeds, the 
image data is evaluated for a stop pattem 1 1 6 at section 1 74. If the test fails, an 
error of invalid stop pattern 182 is returned to scan processor 156. If the test 
succeeds, the decoding process continues. The image data to be decoded is then 
compared with conventional standard Braille codeword values. If comparison test 
1 79 fails, an invalid code fragment error 184 is returned to scan processor 1 56. If 
the test is successful, the decoding process continues to decrypt the image data, 
which is evaluated for additional image data at section 183. If the evaluation 
yields no more image data, decrypted data is returned by section 187 to scan 
processor 156. If the evaluation yields additional image data, the process loops to 
continue decoding of the additional image data until this loop temninates. 

[0043] Referring to Fig. 12, handheld unit 145 decodes a Braille code 159 or 
bar code image 151 . Scanner data 166 is processed by user interi'ace 100 and 
one or more hieroglyphs 19 are displayed that are readable by end-users 203, 
e.g., cognitively disabled, sighted persons who are unable to read Braille or 
standard alphanumeric characters, such as English language words. Hieroglyphs 
19 provide an instruction and direction language for non-text readers which 
includes and is subject to a set of syntax rules that govern the display and 
arrangement of hieroglyphs 19. Display 147 presents hieroglyphs 19 to the end- 
user based on their input, local database record set(s) and programmatic selection 
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structure. The syntax rules control display 147 by specifying graphic content and 
hieroglyph order based upon standard English language-oriented visualizations 
and abstractions, e.g.. to elicit an action on the part of the user by way of a visual 
verb, identification of things or places in end-users 203 physical world, i.e., street 
5 names, addresses, aisles, shelves, signs, etc., and with which end-users 203 
interacts with, or takes action upon. A wide variety of custom hieroglyph symbols 
may be display concepts and logic as graphic content on display 147 in 
accordance with the information being decoded from either Braille symbology code 
1 6 or bar code 151. Of course, it will be understood that other written or spoken 

10 language may also be used to determine the syntax rules that govern the display 
and arrangement of hieroglyphs 19, e.g., Japanese, Chinese, Indo-European 
languages and their root languages, etc. Figures 13 and 14 present a small, 
exemplary portion of a hieroglyph vocabulary used in connection with the present 
invention. This visual language is developed graphically using exaggerated 

15 pantomime, expression, gesture, symbolic posture, motion indicated by "Speed 
lines' , changes in scenery so as to convey changes in location, 

[0044] For example, the "?" hieroglyph is used to present a questioning 
thought, rather than speech, while background images are presented as a part of 
a narrative, so that text in the form of Braille codeword 159 or bar code 151 read 
20 as an image. The visual treatment of words as graphic art forms is part of the 
hieroglyphic vocabulary. Comprehension of an image requires commonality of 
experience, among end-user 203, who will learn to relate this visual graphic 
language through their experience. 
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[0045] Referring to the flow chart of the architecture of the code & 
symbology interpreting software 13 shown in Fig. 15, end-users 203 who are 
sighted, but can not read text due to literacy issues, include end-users with 
cognitive disabilities. End-users 203 engage and interact with system processors 
5 112, e.g., a personal computer, mainframe computer, microcomputer, workstation, 
server, a portable electronic device such as a personal digital assistant (PDA) via 
data entry means 100. Code & symbology interpreting software 13 is 
implemented in the form of a computer software program stored in the memory of 
a system processor device 1 12, although such a system could equivalently be 

10 provided as an electronic circuit or combination of such circuits. Code & 

symbology interpreting software 13 is configured based upon the architecture of 
processing device 112, such as, Client\server, internet, Multi-tier, single user, etc. 
The software components and interfaces are controlled by a processor managers 
216. Task-objects 219 are objects in the environment that end-users 203 must 

15 interact with or take action upon. Task-objects 21 9 are the nouns that are 

depicted by hieroglyphs 19 that drive the human-machine interaction in the form of 
a dialog. Processor managers 216 control data exchange with relational 
databases 218 using SQL commands and, if required by system architecture, 
connectivity resources. Any relational database 218 can be used with the present 

20 invention provided that it accepts SQL commands. Non-limiting examples of 

relational databases that can be used with the present invention include Microsoft 
Access, Microsoft SQL Server, and Oracle. Relational database 218 may be a 
local database or an on-line database. In one embodiment, the database is 
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accessed using Open Database Connectivity (ODBC) by declaring the database 
as tlie data source using a Windows ODBC data source administrator. Processor 
manager 216 may also be part of an internet-based code & symbology interpreting 
software 13 . Connectivity resources 217 for handheld devices using PALM 
5 operating systems include PALM conduits for ODBC connection for database 
synchronization. Windows handheld operating systems also may require ODBC 
connection for database synchronization. Processor manager 216 controls device 
processors 223 through interfaces 221. In one embodiment, interfaces 221 
comprise Windows API. Processor managers 216 control a User interface 222 
10 that communicates visual graphic language 225 to end-users 203 . Display of 
visual graphic language at section 225 is based on end-users 203 input, local 
database record set(s) and programmatic selection structure. 

[0046] Referring to Fig. 16, presents one exemplary embodiment of a task 
module for code & symbology interpreting software 13. Processing device 112 

15 architecture in this embodiment is a personal computer and a scan enabled PDA. 
The connectivity resources include SQL command, ODBC and a handheld 
synchronization method. The embodiment is a simplified illustration of a task 
module configured for a cognitively disabled person to work effectively in an 
inventory control facility at a typical product distribution center. For this 

20 embodiment, processor managers 216 are database manager 230, Inventory 
manager 231 , task manager 233 and scanner manager 239. Each processor 
manager 216 has a controller attribute, and may collaborate within its class. 
Inventory manager 231 collaborates with database manager 230 to process 
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inventory data 232. Task manager 233 collaborates with scanner manager 239 to 
process scanned inputs from end-user 203 . Each manager has specific task 
processes that it controls. Task manager 233 controls a receive processor 234, a 
pick processor 235, a count processor 236 and a QA processor 238. Scanner 
5 manager 239 works with scan decoder 165. As a device processor 223, scanner 
manager 239 checks for and handles device errors and status. As components of 
user interface 222, receive processor 234, pick processor 235, count processor 
236 and QA processor 238 display syntactically arranged hieroglyphs 19 and 
directions to end-user 203. 

10 [0047] Referring to Fig. 1 7, a hand held scanner error handling for code & 
symbology interpreting software 13 comprises scanner manager 239 performing 
an error and status check for each scan attempt request 245. Test 246 evaluates 
scan success. If test 246 fails, scan event codes 247 return an error message 248 
to end-user 203. In this embodiment, rescan graphic 249 is displayed to end user 

15 203. If end-user 203 quits, the application ends at section 250. If end-user 203 
scans another Braille code 159 or bar code 151 , generic reference Bcode, as input 
251, the process loops to test 246 and continues its evaluation sequence. If test 
246 succeeds, a check scanner status 252 is initiated using status event state 
change codes 253, ending the scan 254 returning the result 255, generic 

20 reference, and Bcodevalue to scanner manager 239. 

[0048] Referring to Fig. 18, a graphics process for code & symbology 

interpreting software 13 may be prepared or written with a visual basic 

programming language. However, the present invention can be implemented using 
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a variety of common programming languages such as Visual Basic, C++, C#, 
Java, etc. Using Visual Basic, frmStatusForm 260 is the Windows based user 
interface from which show graphics 261 process runs. Once declare has been set 
as variable intlconNum and x as Integer 262, the "For Next" loop will continue until 
5 intlconNum is greater than x, the highest index numbered graphics file in graphic 
array 266. Within the "For Next" loop, test 264 is initiated to evaluate 
x=intlconNum. If test 264 is true, than the graphic file image is visible on 
frmStatusForm 260. The image is then displayed to end-user 203. If test 264 is 
false, then the graphic image is not visible on fnnStatusForm 260, and the image 

10 is not displayed to end-user 203. The process stops at 268 when the index 

exceeds the available graphics in the array. In this embodiment, graphic array 266 
stores a subset of hieroglyphs 19. This subset includes task specific hieroglyphs 
19 such as pick and receive symbols. Selection of hieroglyphs 19 are based upon 
the task to be accomplished by end-user 203, and the task-objects 219 being 

15 interacted with or acted upon by end-users 203. The show graphics 261 process 
is used with a menu process for end-users 203 to select processors. The show 
graphics 261 process may be called by and launched from other processes as 
well. Show graphics 261 is not a stand-alone process and in another embodiment, 
no graphics array 266 is used. 

20 [0049] Referring to Fig. 19, is a relational database structure for code & 
symbology interpreting software 13 which includes relational databases 218 to 
store bar code 151 and Braille code 159 values for use by processor managers 
216. A handheld scanner reads barcode 151 or Braille code 159 values and 
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enters them into a required field based on a particular task module. These values 
may be assigned an equivalent name based on the task module containing it. 
These fields may be primary key fields for the database table. 

[0050] In Warehouse management, fulfillment management or inventory 
5 control modules, a bar code 1 51 value for each product can be a SKU (stock 
keeping unit), product code, UPC code, or other scanner readable product 
identification value. The quantity field may be called item quantity, Quantity In 
Warehouse or quantity. For this embodiment of the present invention, in records 
and warehouse management modules, the primary key fields are the WHAddress 

10 and RFAddress. These fields store the text values of scanner data. For 
illustration, the WHAddress and RFAddress are primary key fields in the 
warehouse location and records location tables respectively. They are text fields 
of fixed size. They are also populated to give a specific and unique address to 
every location in the database. For every record, there must be a Braille coded 

15 address in the physical warehouse or record storage facility. End-users 203 will 
scan Braille codes 159 and use the navigator processor and user interface 100, to 
navigate their course in the physical warehouse space. The navigator processor 
shows end-user 203 where to go within the physical warehouse space based on 
the particular Braille codes 159 that have been scanned. Processor managers 

20 216, relational databases 218, and the task-objects 219 form the dialog 21 5 of 
hieroglyphs 19 that instructs and guides end-users 203. 

[0051] Depending upon the architecture of processing system device 1 12, 
relational databases 218 may reside on a server and\or a mobile tier. In this 
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embodiment, relational databases 218 for inventory control. Virtual Manifest 274 
reside on a personal computer 112 and a pocket manifest 275 resides on a 
handheld computer. Connectivity resource 277 facilitates synchronization of 
database tables. Processor manager 216 requires database information at the 
start of a process that tests for the existence of the database. Start 270 begins 
test 271 , making a machine code inquiry as to whether the required database 
exists? If test 271 is answered yes, then the process ends at section 276. If test 
271 is answered no, then the required database is created with the required tables 
and fields. In this embodiment, the required database is the Virtual Manifest 274. 

[0052] Referring to Fig. 20, a count processor for the code & symbology 
interpreting software 13 includes show graphics 261 to process with a menu 
process to display a graphic menu 280. End-user 203 taps a touch receptive 
handheld display screen 147 to select a count processor, so as to Declare 
CountVaraible as Integer 282. A Launch process 283 checks for the required 
database (Fig. 19) and the initiator opens the pocket manifest 284 database. 
Next, end-user 203 is prompted with one or more hieroglyphs 19 to scan a first 
item. End-user scans a bar code 151 or Braille Code 159 to start count 286. Upon 
scan attempt, scan error processor 287 engages scanner manager 239 process 
(Fig. 17). If the scan is successful and end-user 203 does not quit the application, 
then SQL query 288 retrieves SKU records matching the "%BarcodeValue%". 
Test 289 evaluates the result of this query. If test 289 fails to match SKU records, 
then the international symbol for "no" is displayed 293 to end-user 203. The 
"%BarcodeValue%" is processed by QA monitor 292. The OA monitor adds the 
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unmatched value to a table to log it as a possible end-user 203 error for review. 
The count processor continues allowing end-user 203 the choice of continuing or 
ending the application at test 291 . If test 291 is answered yes, end-user 203 
scans another item and the process loops to 286. If test 291 is answered no, end- 
5 user 203 quits, and the count processor ends at 295. If test 289 succeeds, end- 
user 203 is instmcted to continue counting the item by displaying Count graphic 
290. Business mies and model 294 that govern the task assignments of end- 
users 203 are used to sequence graphics 295 using code & symbology 
interpreting software 13 syntax, graphic nouns and verbs. Using the count 
10 processor, end-users 203 follow the graphic sequence of one or more syntactically 
arranged hieroglyphs 19 complete task assignments such as item counts, 
inventory stocking, and inventory audit. 

[0053] Non-limiting examples of other task modules configurable using this 
invention are retail store stocking and shopping, records management, order 
15 fulfillment, warehouse management, navigating aisles & streets, reading signs etc. 
Various task modules may be configured based on the needs of the end-users 
203. 

[0054] It is to be understood that the present invention is by no means 
limited only to the particular constructions herein disclosed and shown in the 
20 drawings, but also comprises any modifications or equivalents within the scope of 
the claims. 
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